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FOREWORD 


The  Infantry  Forces  Research  Unit  of  the  U.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences  conducts  research  that  contributes  to  a  better  understanding  of 
soldier-based  instructional  issues  under  its  Training  Modernization  for  Infantry  Forces  research 
program.  In  support  of  this  objective,  our  scientists  have  participated  in  14  field  experiments 
conducted  under  the  auspices  of  the  Military  Operations  in  Urban  Terrain  (MOUT)  Advanced 
Concept  Technology  Demonstration  (ACTD)  program.  The  last  of  these  field  experiments 
formed  the  basis  of  this  report.  It  investigated  the  degree  to  which  squad  radios  contribute  to  the 
battlefield  situation  awareness  (SA)  of  squad  leaders  and  squad  members. 

Prior  to  this  field  experiment,  we  developed  a  pair  of  new  instruments  for  measuring  the 
SA  of  small  unit  leaders  based  on  information  that  could  be  gleaned  from  the  unobtrusive  real¬ 
time  monitoring  of  platoon  and  squad  radio  networks.  From  an  initial  pool  of  318  critical 
incidents  of  communication  behavior,  60  were  culled  on  the  basis  of  high  independent  evaluator 
agreement  to  form  the  Radio  Communications  Checklist  of  Leader  Awareness  (RCCOLA)  and 
the  Future  Expectations  of  Likely  Leader  Awareness  (FELLA)  scale.  This  report  documents  the 
methodological  development  of  these  new  SA  measures  and  presents  the  results  of  a  limited  field 
evaluation  involving  seven  squad  leaders  for  six  trials  each. 

Although  the  size  of  our  squad  leader  sample  prevents  one  from  drawing  any  definitive 
conclusions  about  the  RCCOLA  and  FELLA  instruments,  results  were  promising  and  generally 
positive.  Both  instruments  demonstrated  high  levels  of  interrater  agreement  and  both  can  be  said 
to  possess  a  certain  measure  of  content-related  validity,  based  on  the  way  the  instruments  were 
constructed.  We  recommend  their  future  use,  for  both  training  and  continued  research,  in  all 
situations  where  small  unit  radio  networks  can  be  monitored  by  qualified  personnel.  These 
results  have  been  presented  to  key  research  sponsors  in  the  MOUT  ACTD  and  Objective  Force 
Warrior  (OFW)  programs  during  March  of  2003.  We  hope  to  interest  you  to  learn  more  about  the 
potential  SA  benefits  of  improved  soldier  communication  at  the  small  unit  level. 


KATHLEEN  A.  QUINKERT 
Acting  Technical  Director 
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DEVELOPMENT  AND  EVALUATION  OF  COMMUNICATION-BASED  MEASURES 
OF  SITUATION  AWARENESS 


EXECUTIVE  SUMMARY 


Research  Requirement: 

It  is  generally  accepted  that  the  most  precise  way  of  gauging  situation  awareness  (SA)  is 
to  use  objective  measures,  where  important  aspects  of  the  situations  investigated  can  be 
accurately  known  to  both  research  participants  and  experimenters.  Unfortunately,  objective 
measures  are  relatively  obtrusive  and  difficult  to  administer.  They  also  lack  robustness  across 
situations,  as  a  new  test  must  be  developed  for  each  experimental  scenario.  The  present 
investigation  sought  to  develop  and  field  test  two  new  behavioral  measures  of  SA  that  were 
based  on  the  content  of  small  unit  radio  transmissions  monitored  by  unobtrusive  observers. 


Procedure: 

A  team  of  four  retired  military  personnel  was  given  a  two-hour  training  workshop  on  how 
to  write  critical  incidents  of  communication  behavior.  Over  the  course  of  the  next  several  weeks, 
the  team  generated  a  pool  of  318  incidents  of  communication  behavior,  each  intended  to 
represent  either  an  outstanding,  typical,  or  poor  level  of  SA  on  the  part  of  squad  and  platoon 
leaders.  This  item  pool  was  then  given  to  a  group  of  24  independent  evaluators.  Each  evaluator 
was  asked  to  judge  whether  or  not  each  item  reflected  the  SA  of  small  unit  leaders.  If  they 
judged  an  item  as  being  related  to  the  concept  of  SA,  they  were  also  asked  to  indicate  whether 
the  item  suggested  an  outstanding,  typical,  or  poor  level  of  SA.  For  each  of  the  three  levels  of 
SA,  the  20  items  with  the  highest  levels  of  agreement  among  the  independent  evaluators  were 
formed  into  two  communication-based  SA  measures.  These  were  termed  the  Radio 
Communications  Checklist  of  Leader  Awareness  (RCCOLA)  and  the  Future  Expectations  of 
Likely  Leader  Awareness  (FELLA)  scale.  Although  both  measures  were  based  on  the  same  60 
items,  they  differed  greatly  in  their  format.  The  RCCOLA  checklist  was  used  to  record  how 
frequently  a  specific  set  of  behaviors  occurred  during  the  course  of  a  mission,  while  the  FELLA 
questionnaire  was  completed  at  the  end  of  a  mission  and  asked  raters  to  predict  or  estimate  the 
likelihood  these  behaviors  would  occur  in  future  missions  along  a  seven-point  Likert  scale. 


Six  field  trials  were  then  conducted  with  each  of  seven  squad  leaders  and  their  respective 
squads.  All  trials  for  a  given  squad  consisted  of  variations  of  a  reconnaissance/link-up  mission 
and  were  completed  within  a  12-hour  time  period.  Three  squad  radio  conditions  were  used,  with 
one  day  and  one  night  trial  conducted  for  each  condition.  The  conditions  were  no  squad  radio, 
squad  radio  with  only  the  squad  leader  transmitting,  and  squad  radio  with  both  the  squad  leader 
and  squad  members  having  the  freedom  to  transmit.  Based  on  their  monitoring  of  squad  and 
platoon  radios,  two  independent  raters  completed  a  separate  RCCOLA  checklist  during  each  of 
the  42  total  trials,  as  well  as  a  separate  FELLA  scale  after  the  completion  of  each  trial. 
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Findings: 


Based  on  the  percentage  of  nearly  identical  item  scores  (+/- 1),  interrater  agreement  was 
found  to  be  generally  high  for  both  the  RCCOLA  checklist  (97.5%)  and  the  FELLA  scale 
(84.4%)  across  trials.  Because  they  were  constructed  from  items  having  a  high  level  of 
agreement  among  independent  evaluators,  we  can  also  assume  both  measures  possess  a  certain 
amount  of  content-related  validity.  Though  it  did  not  approach  statistical  significance,  a  small 
consistent  trend  among  the  squad  radio  conditions  and  our  communication-based  measures  was 
found.  In  particular,  the  highest  average  SA  scores  were  obtained  during  trials  when  squad 
members  were  allowed  to  transmit  freely,  perhaps  because  there  were  a  significantly  greater 
number  of  audible  squad  radio  transmissions  during  these  trials  (p  <  .001). 


Utilization  of  Findings: 

We  recommend  the  use  of  RCCOLA  checklist  and  FELLA  scale  in  future  research,  field 
exercises,  and  virtual  training  environments  where  radio  transmissions  of  small  unit  personnel 
can  be  monitored.  Additional  research  is  needed  to  increase  the  sample  of  small  unit  leaders,  to 
determine  the  relationship  of  communication-based  measures  to  more  objective  measures  of  SA, 
and  to  gauge  their  utility  for  a  wider  variety  of  soldier  missions. 

As  squad  and  platoon  radios  are  beginning  to  proliferate  within  small  units,  the  ability  of 
our  soldiers  to  communicate  effectively  with  each  other  will  become  a  critical  factor  influencing 
their  ultimate  level  of  combat  effectiveness.  How  squad  members  contribute,  or  fail  to 
contribute,  to  the  SA  of  their  squad  and  platoon  leaders  is  an  after-action  review  topic  that  needs 
more  emphasis.  Similarly,  we  also  need  to  understand  and  emphasize  how  the  communication 
behaviors  of  small  unit  leaders  contribute  or  detract  from  the  situational  understanding  of  their 
subordinates.  Before  the  promise  of  better  small  unit  communication  can  be  realized,  however, 
we  must  get  trainers  and  observer/controllers  to  routinely  monitor  squad  and  platoon  radios 
during  field  exercises  and  training  center  rotations.  Once  optimal  squad  and  platoon  radio 
communication  procedures  have  been  identified,  they  need  to  be  formally  introduced  into 
appropriate  institutional  courses  for  the  benefit  of  junior  leaders.  We  believe  that 
communication-based  measures  of  SA,  such  as  those  explored  in  the  present  report,  can  serve  an 
important  role  in  improving  the  communication  practices  and  resulting  levels  of  situational 
understanding  among  all  soldiers  at  the  small  unit  level. 
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DEVELOPMENT  AND  EVALUATION  OF  COMMUNICATION-BASED 
MEASURES  OF  SITUATION  AWARENESS 


Introduction 

The  present  research  effort  is  a  largely  serendipitous  product  of  two  ongoing  lines  of 
investigation,  specifically  research  related  to  the  measurement  and  training  of  situation 
awareness  (SA)  in  Infantry  leaders  and  research  aimed  at  discovering  the  most  effective  tactics, 
techniques,  and  procedures  (TTPs)  to  use  with  squad  and  platoon  radio  networks.  In  the  first 
case,  identifying  SA  requirements  and  their  relationships  with  emerging  technology  was  the 
focus  of  early  thought  in  the  area  of  Infantry  SA  (Graham  &  Matthews,  1999).  Building  upon  the 
earlier  theoretical  work  of  Endsley  (1995b),  an  Infantry-focused  model  of  individual  SA  was 
developed  (Endsley  et  al.,  2000).  As  a  prelude  to  developing  an  Infantry  SA  trainer,  the  highest- 
priority  S  A  training  requirements  were  also  identified.  These  requirements  were  schema  training, 
task  management  and  prioritization,  communications  training,  and  contingency  planning  (Strater, 
Jones,  &  Endsley,  2001). 

It  is  generally  accepted  that  the  most  precise  way  of  gauging  SA  is  to  use  objective 
measures,  where  various  aspects  of  the  situations  investigated  can  be  accurately  known  to  both 
research  participants  and  experimenters  (Endsley,  1995a;  Redden  &  Blackwell,  2001).  There  the 
focus  of  measurement  is  on  the  degree  to  which  individual  perceptions  of  situational 
characteristics  differ  from  what  is  known  to  be  "ground”  truth.  Though  their  reliability  and 
validity  are  usually  more  than  adequate,  objective  measures  are  relatively  obtrusive  and  difficult 
to  administer.  They  also  lack  robustness  across  situations  (i.e.,  a  new  test  must  be  developed  for 
each  experimental  scenario).  Together  with  the  present  investigation,  some  recent  research 
efforts  have  explored  the  use  of  more  subjective  alternatives  to  traditional  methods  of  measuring 
SA  objectively  (Matthews,  Beal,  &  Pleban,  2002;  Strater,  Endsley,  Pleban,  &  Matthews,  2001). 

In  the  second  case,  while  attempting  to  categorize  and  quantify  radio  transmissions  in 
previous  investigations  of  the  relationship  between  various  small-unit  radio  TTPs  and  SA  (Christ 
&  Evans,  2001;  Dismounted  Battlespace  Battle  Lab  &  Marine  Corps  Warfighting  Lab,  1999; 
Redden  &  Blackwell,  2001),  observers  noticed  that  some  squad  leaders  appeared  to  have  greater 
SA  than  others,  based  solely  on  the  nature  and  content  of  their  radio  communication  with 
superiors,  peers,  and  subordinates.  The  present  research  effort  is  an  attempt  to  quantify  this 
earlier  behavioral  observation.  We  seek  to  eventually  develop  communication-based  measures  of 
SA  that  will  reliably  differentiate  the  performance  of  small  unit  leaders  and  will  closely  mirror 
the  psychometric  properties  of  more  objective  SA  measures. 

The  present  report  is  organized  around  two  separate,  though  interrelated,  research  efforts. 
First,  we  document  the  methodological  development  of  two  communication-based  SA  measures 
from  a  common  pool  of  over  300  behavioral  incidents.  These  measures  are  a  behavioral  checklist 
called  the  Radio  Communications  Checklist  of  Leader  Awareness  (RCCOLA)  and  a 
questionnaire  called  Future  Expectations  of  Likely  Leader  Awareness  (FELLA).  Second,  we 
present  the  results  of  a  limited  number  of  field  trials  of  these  measures  that  were  conducted  with 
Infantry  squad  leaders  and  their  squads.  In  the  last  section  of  the  report  we  discuss  future 
potential  uses  of  the  two  scales. 
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Development  of  Situation  Awareness  Measures 

Our  approach  to  developing  communication-based  SA  measures  involved  the  use  of  the 
critical  incident  technique,  originally  described  by  Flanagan  (1954).  Four  retired  military 
personnel  were  given  a  two-hour  training  workshop  on  how  to  write  suitable  incidents  of 
communication  behavior.  This  training  stressed  four  characteristics  of  critical  incidents:  their 
behavioral  nature,  specificity,  ability  to  differentiate  among  people,  and  clarity.  Examples  of 
both  acceptable  and  unacceptable  incidents  were  provided.  The  first  few  incidents  written  by 
each  participant  were  critiqued  and  suggestions  for  improvement  were  offered.  Over  a 
subsequent  period  of  several  weeks,  participants  generated  a  pool  of  3 1 8  behavioral  incidents  on 
their  own,  each  intended  to  represent  either  outstanding,  typical,  or  poor  SA  on  the  part  of 
platoon  or  squad  leaders.  In  addition  to  using  their  own  experience,  the  four  item  authors 
consulted  a  variety  of  publications  to  stimulate  thought  about  item  content  and  to  widen  the 
conceptual  coverage  of  the  item  pool.  These  publications  included  doctrinal  manuals 
(Department  of  the  Army,  1992, 1994),  the  results  of  a  recent  SA  requirements  analysis  (Strater 
et  al.,  2001),  various  bulletins  and  reports  of  the  Center  for  Army  Lessons  Learned,  as  well  as 
Combat  Training  Center  compendia. 

After  editing  to  create  a  common  style,  the  318  behavioral  incidents  were  given  to  a 
group  of  24  independent  evaluators.  Each  evaluator  was  asked  to  judge  whether  or  not  each  item 
reflected  the  SA  of  small  unit  leaders.  If  they  judged  an  item  as  being  related  to  the  concept  of 
S  A,  they  were  also  asked  to  indicate  whether  the  item  suggested  outstanding,  typical,  or  poor 
SA.  The  questionnaire  used  in  the  independent  evaluation  is  shown  in  Appendix  A.  The  group  of 
independent  evaluators  included  active  duty  military  personnel  (n  —  9),  retired  military  personnel 
(n  =  7),  and  civilian  scientists  familiar  with  military  field  research  (n  -  8).  Table  1  shows  the 
level  of  agreement  on  item  content  between  the  item  authors  and  the  independent  evaluators. 
Generally,  items  written  to  reflect  poor  or  typical  levels  of  leader  SA  were  viewed  as  such  by  a 
majority  of  the  independent  evaluators.  However,  items  written  to  reflect  outstanding  SA  were, 
more  often  than  not,  seen  as  reflecting  a  typical  level  of  SA.  This  effect  was  strongest  among  the 
active  duty  military  evaluators. 


Table  1 

Number  of  Items  Classified  by  Item  Authors  and  Independent  Evaluators 
as  Outstanding,  Typical,  and  Poor  SA 


Intent  of  Item  Authors 

Outstanding 

Typical 

Poor 

Majority  Rating 
of  Independent 
Evaluators 

28 

1 

1 

Typical 

91 

60 

11 

Poor 

1 

20 

105 

Total 

120 

81 

117 

23.33 

74.07 

89.74 
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After  the  results  of  the  independent  evaluation  were  tabulated,  the  318  items  were  sorted 
according  to  their  level  of  evaluator  agreement,  from  highest  to  lowest.  There  were  28  items  that 
fewer  than  20  independent  evaluators  thought  reflected  some  level  of  SA.  These  were  eliminated 
from  any  further  consideration.  An  additional  108  items  were  eliminated  because  the  majority  of 
evaluators  disagreed  with  the  intent  of  the  item  authors.  For  example,  17  of  the  evaluators 
thought  item  40  represented  an  outstanding  level  of  leader  SA,  while  six  thought  it  represented  a 
typical  level  of  S  A,  and  one  was  not  convinced  it  represented  S  A  at  all.  This  item  was  dropped 
because  the  authors  had  intended  it  to  reflect  a  typical  level  of  SA.  For  each  SA  level,  the  20 
items  having  the  greatest  independent  evaluator  agreement  were  selected  for  inclusion  in  the 
RCCOLA  and  FELLA  scales.  Agreement  statistics  for  the  60  chosen  items  are  shown  in  Table  2. 


Table  2 

Mean  Percentage  of  Evaluator  Agreement  for  20  Items  Representing 
Each  of  Three  Levels  of  SA 


Item  SA  Level 

Mean  %  of  Evaluators  Who  Thought 
Items  Related  to  SA 

Mean  %  of  Evaluators  Who  Agreed 
with  Author's  Intent  about  S  A  Level 

Outstanding 

93.12 

63.33 

Typical 

91.87 

82.92 

Poor 

91.46 

88.96 

Radio  Communications  Checklist  of  Leader  Awareness 

As  shown  in  Appendix  B,  the  60  items  selected  as  a  result  of  the  independent  evaluation 
were  formed  into  a  behavioral  checklist  titled  the  Radio  Communications  Checklist  of  Leader 
Awareness  (RCCOLA).  This  SA  measure  was  designed  to  enable  observers  to  record  the 
occurrence  of  SA-related  communication  behaviors  in  real  time  while  listening  to  the  squad  and 
platoon  radio  networks  of  a  squad  leader.  Similarly,  RCCOLA  items  were  designed  to  be 
suitable  for  the  assessment  of  platoon  leader  SA,  by  listening  to  company  and  platoon  radio 
networks.  Every  time  a  particular  behavior  was  heard,  observers  or  raters  would  place  a 
checkmark  next  to  that  item  on  the  checklist.  Thus,  it  was  possible  for  each  RCCOLA  item  to 
receive  multiple  checkmarks,  one  for  each  occurrence  of  a  particular  behavior. 

Items  representing  outstanding,  typical,  and  poor  S  A  were  segregated  and  then  grouped 
into  four  temporal  categories:  planning/preparing,  movement,  actions  on  enemy  contact,  and 
miscellaneous.  These  categories  were  chosen  to  hasten  the  ratings  process  under  real-time 
conditions,  making  it  easier  for  observers  to  locate  particular  items  as  a  mission  unfolds.  The 
categories  were  not  thought  to  be  underlying  factors  of  an  SA  construct.  The  miscellaneous 
category  included  items  that  could  occur  at  any  time  during  a  mission.  For  each  trial  or  mission, 
the  RCCOLA  measure  is  scored  as  follows: 
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number  of  outstanding  checkmarks  -  number  of  poor  checkmarks 
total  number  of  outstanding,  typical,  and  poor  checkmarks 

Possible  RCCOLA  scores  can  range  from  -1  to  +1,  with  a  score  of  0  indicating  a  typical  level  of 
SA  for  squad  leaders. 


Future  Expectations  of  Likely  Leader  Awareness 

The  same  60  items  selected  from  the  independent  evaluation  were  also  formed  into  what 
we  called  the  Future  Expectations  of  Likely  Leader  Awareness  (FELLA)  scale  (see  Appendix  C). 
Unlike  the  RCCOLA  measure,  the  FELLA  scale  was  designed  to  be  completed  at  the  end  of  an 
experimental  trial  or  operational  mission.  Further,  the  FELLA  items  were  listed  in  random  order, 
without  grouping  them  into  categories.  The  FELLA  scale  was  designed  to  be  more  future 
oriented  and  subjective  than  the  RCCOLA  measure,  asking  raters  their  expectations  of  the 
likelihood  of  particular  leader  behaviors  occurring  in  subsequent  missions. 

The  60  FELLA  items  were  scored  on  a  seven-point  Likert  scale  ranging  from  "Highly 
Unlikely"  (1)  to  "Highly  Likely"  (7),  with  the  scale's  midpoint  being  "Hard  to  Say"  (4).  The  20 
items  representing  a  poor  level  of  leader  SA  were  reverse  scored.  An  overall  FELLA  scale  score 
was  obtained  by  calculating  the  mean  of  the  60  items.  Thus,  possible  overall  scores  could  range 
from  1  to  7. 


Global  SA  Assessment  Item 

Finally,  a  global  assessment  item  was  added  to  the  end  of  the  FELLA  scale.  This  item 
asked  raters  about  their  overall  expectations  of  a  leader's  S  A  level  in  future  missions.  The  global 
assessment  item  was  scored  on  a  seven-point  Likert  scale  ranging  from  "Poor"  (1)  to 
"Outstanding"  (7),  with  "Center  of  Mass"  being  the  midpoint  anchor.  If  a  large  enough  leader 
sample  could  be  obtained  in  future  experimentation,  this  item  could  help  to  determine  the 
relative  contribution  of  particular  leader  behaviors  to  the  overall  concept  of  S  A. 


Evaluation  of  Situation  Awareness  Measures 

Field  trials  using  the  RCCOLA  checklist  and  the  FELLA  scale  were  conducted  as  part  of 
a  larger  field  experiment  investigating  the  degree  to  which  squad  radios  enhanced  soldier  SA. 
Sponsored  by  the  Military  Operations  in  Urban  Terrain  (MOUT)  Advanced  Concept  Technology 
Demonstration  (ACTD)  program,  this  experiment  was  a  direct  follow-on  effort  to  an  earlier 
experiment  whose  results  have  already  been  reported  in  detail  (Christ  &  Evans,  2002;  Redden  & 
Blackwell,  2001).  The  earlier  experiment  involved  the  use  of  experienced  squads  conducting 
offensive  and  defensive  missions  in  an  urban  environment,  where  squad  members  maintained 
direct  visual  contact  with  each  other  much  of  the  time.  In  contrast,  the  present  experiment 
involved  the  use  of  relatively  inexperienced  squads  conducting  reconnaissance  and  link-up 
missions  in  a  largely  wooded  environment,  where  the  squad's  two  teams  were  geographically 
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separated  from  each  other  most  of  the  time.  Each  team  approached  a  fenced  compound  from 
different  directions,  where  they  were  instructed  to  surreptitiously  report  any  activities  of 
observed  enemy  and  civilian  personnel.  Later,  the  teams  linked  up  at  a  designated  checkpoint, 
which  served  as  another  site  for  the  observation  and  reporting  of  enemy  and  civilian  activities. 

Research  Participants 

Research  participants  were  seven  squad  leaders,  each  having  two  teams  of  either  three  or 
four  men  each.  Most  of  these  squad  leaders  were  relatively  inexperienced,  either  having  been 
recently  assigned  to  the  squad  or  having  been  assigned  to  a  temporary  squad  leadership  position. 
In  some  cases,  squad  leaders  had  no  prior  field  training  experience  with  their  squads.  Our 
research  focused  only  on  the  SA  of  the  squad  leaders,  as  reflected  in  their  radio  communications 
with  squad  members  and  with  a  simulated  platoon  leader.  The  platoon  leader's  role  was  played 
by  an  experimenter  whose  outgoing  radio  transmissions  were  largely  dictated  by  a  rehearsed 
mission  script  for  each  trial. 

Measures  and  Raters 

A  team  of  two  retired  Infantry  officers  served  as  independent  raters,  each  completing  a 
separate  RCCOLA  checklist  during  each  trial,  as  well  as  a  separate  FELLA  scale  and  global 
assessment  item  immediately  after  the  completion  of  each  trial.  One  member  of  the  rating  team 
participated  in  rating  all  seven  squad  leaders.  The  second  member  of  the  rating  team  rated  only 
the  first  three  squad  leaders.  A  third  retired  Infantry  officer  served  as  the  second  rater  for  the  last 
four  squad  leaders.  Finally,  a  civilian  member  of  the  research  staff  separately  logged  the  number 
and  types  of  radio  transmissions  heard  over  the  squad  and  platoon  radio  networks  using  a  real¬ 
time  categorization  scheme  described  by  Christ  and  Evans  (2002).  Specifically,  each  squad  and 
platoon  radio  transmission  was  logged  into  one  of  1 5  mutually  exclusive  categories:  Provide 
Acknowledgment,  Provide  Direction,  Provide  Information  (Friendly),  Provide  Information 
(Threat),  Provide  Opinion,  Request  Acknowledgment,  Request  Direction,  Request  Information 
(Friendly),  Request  Information  (Threat),  Request  Opinion,  Unrelated  to  Mission, 
Administrative/Other,  Inaudible,  Break  Squelch,  or  Hot  Microphone. 

Procedure 

Each  of  the  seven  squad  leaders  completed  six  trials.  Consequently,  each  SA  measure 
was  used  during  each  of  the  42  separate  trials.  The  average  duration  of  each  trial  was  41  minutes, 
with  the  reconnaissance  portion  requiring  an  average  of  20  minutes  and  the  link-up  portion 
requiring  an  average  of  21  minutes.  Each  squad  leader's  first  three  trials  were  conducted  during 
daylight  hours  and  their  last  three  trials  were  conducted  at  night.  All  trials  for  an  individual 
squad  leader  were  completed  over  a  12-hour  period.  For  scheduling  reasons,  the  order  of  day  and 
night  trials  was  not  counterbalanced  across  squad  leaders,  so  visibility  level  was  not  a  factor 
analyzed  in  this  experiment. 

There  were  three  squad  radio  conditions  evaluated  in  the  experiment,  with  each  day  and 
night  trial  having  a  different  squad  radio  condition.  One  day  trial  and  one  night  trial  for  each 
squad  leader  was  conducted  without  squad  radios.  In  this  baseline  condition,  the  squad  leader 
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used  a  radio  only  to  communicate  with  the  platoon  leader.  On  all  other  trials,  the  squad  leader 
had  two  radios,  one  to  communicate  with  his  squad  members  and  the  other  to  communicate  with 
the  platoon  leader.  In  the  second  squad  radio  condition,  only  the  squad  leader  could  initiate 
transmissions  over  the  squad  radio.  Squad  members  could  listen  and  acknowledge  the  receipt  of 
a  squad  leader  transmission,  but  could  not  initiate  transmissions  themselves.  In  the  third  squad 
radio  condition,  squad  members  were  free  to  initiate  transmissions  at  any  time,  either  to  their 
squad  leader  or  to  other  squad  members.  The  order  of  these  squad  radio  conditions  was  largely 
counterbalanced  across  trials.  Three  different  Opposing  Force  (OPFOR)  scripts  and  two  different 
lanes  of  movement  were  used  alternately  across  trials,  insuring  that  each  squad  leader 
encountered  a  novel  set  of  circumstances  on  each  of  his  six  trials. 

Results 

Due  to  the  relatively  small  sample  of  squad  leaders  ( n  —  7)  and  the  variability  in  their 
performance  across  trials,  our  analyses  were  based  principally  on  the  use  of  descriptive  statistics. 
As  shown  in  Table  3,  the  performance  of  the  squad  leaders,  averaged  across  raters  and  trials, 
tended  to  fall  within  the  middle  and  lower  ranges  of  our  three  SA  measures. 


Table  3 

Mean  Scores  of  Two  Raters  for  Three  SA  Measures 


SA  Measure 

M 

SD 

Potential  Range 

Actual  Range 

RCCOLA  Checklist 

-0.18 

0.33 

-1.00  to +1.00 

-1.00  to  +0.55 

FELLA  Scale 

4.04 

0.79 

1.00  to  7.00 

2.24  to  5.96 

Global  Assessment  Item 

3.04 

1.38 

1.00  to  7.00 

1.00  to  6.00 

Note.  RCCOLA  =  Radio  Communications  Checklist  of] 

Leader  Awareness.  FELLA  =  Future 

Expectations  of  Likely  Leader  Awareness.  Table  entries  are  based  on  six  trials  for  each  of  seven 
squads. 


The  percentage  of  interrater  agreement  on  our  three  measures  was  found  to  be  generally 
high,  as  shown  in  Table  4.  A  plot  of  RCCOLA  total  scores  for  the  two  raters  across  42  trials  is 
shown  in  Figure  1 .  The  integers  shown  in  the  body  of  Figure  1  are  squad  leader  numbers.  Each 
squad  leader  is  shown  six  times,  once  for  each  trial.  Some  squad  leaders  demonstrated  a  fairly 
consistent  level  of  performance  across  trials  (e.g.,  Squad  Leader  5),  while  others  were  highly 
inconsistent  (e.g.,  Squad  Leader  1). 


Table  4 

Percentage  of  Interrater  Agreement  Across  Items  and  Trials 


S  A  Measure 

N 

%  of  Identical  Item 
Scores  Between  Raters 

%  of  Nearly  Identical 
Item  Scores  (+/-  1) 

RCCOLA  Checklist 

2520 

91.3 

97.5 

FELLA  Scale 

2520 

52.9 

84.4 

Global  Assessment  Item 

42 

59.5 

100.0 

Note.  RCCOLA  =  Radio  Communications  < 

Checklist  of  Leader  Awareness.  FELLA  =  Future 

Expectations  of  Likely  Leader  Awareness, 
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RCCOLA 

4  555  5 

0  Scores  of 

4466 

Rater  1 

1  64  21  7 

-.1 

2 

7  224 

-.2 

1  6  7 

3 

-.3 

7  2 

3 

-.4 

6 

-.5 

1 

6  7  33 

J 

-.6  1 

3 

-.7 

4 

-.8 

-.9 

-1.0  17 

RCCOLA  Scores  of  Rater  2 

-1.0  -.9  -.8  -.7  - 

-.6  -.5  -.4  -.3  -.2  -.1  0  .1  .2  .3  .4  .5  .6  .7 

Figure  1 .  A  comparison  of  the  RCCOLA  scores  obtained  from  two  raters  across  42  trials  (six 
trials  for  each  of  seven  squad  leaders). 

Although  we  had  no  expectation  that  all  RCCOLA  items  could  be  observed  during  the 

course  of  every  conceivabl 

e  type  of  mission,  we  found  that  23  of  the  60  items  (38%)  were  never 

observed  by  either  rater  during  our  field  trials.  These  unobserved  items  included  12  of  the  20 
outstanding  SA  items  (60%),  2  of  the  20  typical  SA  items  (10%),  and  9  of  the  20  poor  SA  items 
(45%).  Most  of  these  unobserved  items  can  be  attributed  to  the  nature  of  the  reconnaissance  and 
link-up  mission  scenarios  used  in  the  present  experiment.  Because  squads  were  expected  to 
observe  the  enemy  without  being  detected  and  to  refrain  from  engaging  them  with  weapons, 

RCCOLA  items  involving  ammunition  resupply,  fire  control,  and  casualty  evacuation  behaviors 
were  unlikely  to  occur.  However,  there  were  also  a  few  RCCOLA  items  that  could  conceivably 
have  occurred,  but  were  just  never  observed  among  the  squad  leaders  in  our  sample  (e.g,,  "treats 
a  sound  recommendation  or  advice  from  subordinates  as  an  interruption  and  takes  no  action"). 
The  frequency  with  which  each  RCCOLA  item  was  observed  is  shown  in  Appendix  D. 

During  the  course  of  data  collection,  raters  noticed  it  was  harder  to  form  an  opinion  about 
a  squad  leader's  level  of  SA  on  baseline  trials,  when  no  squad  radio  was  used.  This  anecdotal 
finding  was  supported  by  a  comparison  of  the  average  number  of  items  receiving  a  rating  of  four 
on  the  FELLA  scale  across  the  experimental  radio  conditions.  When  no  squad  radio  was  used,  45 
of  the  60  items  were  rated  as  "hard  to  say"  by  the  two  raters  on  average.  When  a  squad  radio  was 
used,  with  squad  members  listening  but  not  transmitting,  the  average  number  of  items  rated  as 
"hard  to  say"  fell  to  21.  That  number  was  reduced  even  further,  to  18,  when  squad  members  were 
allowed  to  freely  transmit.  These  overall  differences  across  conditions  were  found  to  be 
statistically  significant,  F(2,12)  =  42.41,  p  <  .0001.  Pairwise  comparisons  using  the  Least 
Significant  Difference  Test  indicated  that  both  squad  radio  conditions  were  significantly 
different  from  the  no  squad  radio  condition  (p  <  .001),  though  the  two  squad  radio  conditions 
were  not  significantly  different  from  each  another. 

Though  it  did  not  approach  statistical  significance,  there  was  a  small  consistent  trend 
among  the  squad  radio  conditions  and  our  three  communication-based  SA  measures  (see  Table 
5).  For  each  measure,  the  highest  SA  scores  were  obtained  during  trials  when  a  squad  radio  was 
used,  particularly  when  squad  members  were  allowed  to  transmit  freely.  In  addition,  significantly 
greater  numbers  of  audible  squad  radio  transmissions  occurred  during  these  trials,  F(l,6)  = 

22.24,  p<.  001. 


Table  5 

Mean  Scores  of  Three  SA  Measures  and  Mean  Number  of  Radio  Transmissions 
for  Three  Squad  Radio  Conditions 


SA  Measure 

Audible 
Squad  Radio 
Transmissions 
M(SD) 

Squad  Radio  Condition 

FELLA 

M(SD) 

No  Squad  Radio 

-0.21  (.49) 

4.00  (.39) 

2.89(1.44) 

none 

Squad  Radio  (only  SL 
could  transmit) 

-0.21  (.08) 

4.01  (.87) 

3.04(1.45) 

37.29  (6.91) 

Squad  Radio  (SL  &  squad 
members  could  transmit) 

-0.11  (.06) 

3.18(1.34) 

124.64  (26.35) 

Note.  RCCOLA  =  Radio  Communications  Checklist  of  Leader  Awareness.  FELLA  =  Future 
Expectations  of  Likely  Leader  Awareness,  There  were  14  trials  for  each  squad  radio  condition, 
one  day  trial  and  one  night  trial  for  each  squad  leader. 
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Discussion 


Although  the  size  of  our  squad  leader  sample  limits  the  conclusions  one  can  draw  from 
the  field  trials,  the  findings  obtained  were  largely  positive.  Our  communication-based  measures 
of  SA  appear  to  have  a  sufficient  level  of  interrater  agreement.  Additionally,  they  possess  a 
certain  measure  of  content-related  validity  based  on  the  way  they  were  constructed  (i.e.,  SMEs 
had  to  agree  that  items  reflected  leader  SA  in  order  to  be  included  on  the  RCCOLA  checklist  and 
FELLA  scale).  The  measures  also  appear  sensitive  to  differences  in  the  way  squad  radios  were 
used  (e.g.,  whether  or  not  squad  members  were  allowed  to  transmit).  For  these  reasons,  we 
recommend  the  use  of  the  RCCOLA  checklist  and  FELLA  scale  in  future  research,  field 
exercises,  and  virtual  training  environments  where  radio  transmissions  of  small  unit  personnel 
can  be  monitored.  Further  research  is  needed  to  increase  the  sample  of  small  unit  leaders,  to 
determine  the  relationship  of  communication-based  measures  to  more  objective  measures  of  SA, 
and  to  gauge  their  utility  for  a  wider  variety  of  soldier  missions. 

Raters  had  more  difficulty  using  the  FELLA  scale  than  they  did  the  RCCOLA  checklist. 
The  FELLA  scale  required  a  greater  number  of  discrete  decisions  to  be  made  and  it  required  the 
rater  to  estimate  or  predict  the  likelihood  of  future  events  based  on  events  in  the  recent  past. 

These  decisions  and  estimates  had  to  be  made  within  a  time  limit  of  approximately  15  minutes, 
indirectly  imposed  by  the  pace  with  which  experimental  trials  were  run.  In  contrast,  the 
RCCOLA  checklist  only  required  the  rater  to  note  how  often  specific  behaviors  occurred  during 
the  course  of  a  mission.  The  FELLA  scale  was  also  more  difficult  to  use  because  the  wording  of 
several  of  its  items  was  found  to  be  confusing  (e.g.,  was  it  highly  unlikely  "this  SL  could  be 
expected  not  to  report  being  in  a  danger  area  or  could  be  expected  to  take  no  action  to  avoid 
it?").  Future  use  of  the  FELLA  scale  should  consider  small  wording  changes  to  help  clarify  the 
intended  meaning  of  these  problematic  items,  even  if  the  integrity  of  the  original  wording  has  to 
be  compromised.  While  the  FELLA  scale  was  intended  to  be  more  of  a  research  tool,  for  use  in 
item  analysis  and  factor  analytic  studies  of  the  instrument’s  underlying  content  and  structure,  the 
FELLA  scale  could  also  be  used  in  situations  where  raters  cannot  rely  on  real-time  methods  of 
leader  evaluation. 

Based  on  informal  conversations  held  with  squad  leaders  before  their  first  trial,  we  knew 
our  sample  was  not  representative  of  Infantry  squad  leaders  overall.  In  particular,  our  sample  was 
notably  inexperienced.  As  a  result,  the  performance  of  many  squad  leaders  was  erratic  over  trials 
and  the  upper  levels  of  the  communication-based  SA  measures  were  rarely  used.  Future  research 
with  these  measures  should  seek  to  broaden  the  squad  leader  sample  in  terms  of  their  experience, 
as  well  as  extend  the  sample  to  platoon  leaders,  a  group  for  which  most  of  our  behavioral  items 
should  still  apply.  It  should  also  be  noted  that  the  methods  used  to  create  the  SA  instruments 
described  in  the  present  report  could  be  easily  applied  to  other  occupations  where  personnel 
routinely  communicate  via  radio  (e.g.,  police,  fire,  and  paramedic  organizations). 

As  squad  and  platoon  radios  become  more  common  items  of  equipment  within  small 
units,  the  ability  of  our  soldiers  to  communicate  effectively  with  each  other  will  become  a  critical 
factor  influencing  their  ultimate  level  of  combat  effectiveness.  In  the  past,  when  most  small  unit 
personnel  did  not  communicate  with  radios,  this  was  an  issue  rarely  addressed  in  after-action 
reviews  (AARs)  of  unit  performance.  This  situation  needs  to  change  in  a  hurry.  How  squad 
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members  contribute,  or  fail  to  contribute,  to  the  SA  of  their  squad  and  platoon  leaders  is  an  AAR 
topic  that  needs  more  emphasis.  Similarly,  we  also  need  to  understand  and  emphasize  how  the 
communication  behaviors  of  small  unit  leaders  contribute  or  detract  from  the  situational 
understanding  of  their  subordinates.  Before  the  promise  of  better  small  unit  communication  can 
be  realized,  however,  we  must  get  trainers  and  observer/controllers  to  routinely  monitor  squad 
and  platoon  radios  during  field  exercises  and  training  center  rotations.  Once  optimal  squad  and 
platoon  radio  communication  procedures  have  been  identified,  they  need  to  be  formally 
introduced  into  appropriate  institutional  courses  for  the  benefit  of  junior  leaders.  We  believe  that 
communication-based  measures  of  SA,  such  as  those  explored  in  the  present  report,  can  serve  an 
important  role  in  improving  the  communication  practices  and  resulting  levels  of  situational 
understanding  among  all  soldiers  at  the  small  unit  level. 
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Appendix  A 

Independent  Evaluation  Questionnaire 
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The  purpose  of  this  exercise  is  to  employ  your  expertise  in  identifying  and  rating  audible  leader  behaviors  that  could  be  used  to 
evaluate  leader  situation  awareness  (SA).  Each  item  in  the  following  list  is  a  platoon  or  squad  leader  behavior  that  might  conceivably 
be  heard  over  the  course  of  one  or  more  radio  transmissions.  Your  task  is  to  rate  each  item  according  to  how  well  it  reflects  a  leader's 
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OPORD  Operations  Order 

SITREP  Situation  Report 

SOP  Standing  Operating  Procedure 


__  _ _ _ _ _____  Behavior _  Poor  Typical  ,  Outstanding 

After  receiving  a  change  in  mission,  the  leader  announces  a  movement  formation  or  movement  technique 

inconsistent  with  the  reported  likelihood  of  enemy  contact  and/or  need  for  speed. _ I _ 

At  the  request  of  the  higher  element,  the  leader  does  not  report  vulnerability  to  enemy  activity  in  his  area. _ _ _ 

In  order  to  provide  enemy  information  to  the  higher  element,  the  leader  requests  information  from  subordinates. 


Leader  cannot  be  reached  on  the  radio  during  enemy  contact,  despite  repeated  attempts  to  contact  him. _ 

Leader  changes/adjusts  movement  formations  based  on  likelihood  of  enemy  contact. _ _ 

Leader  changes/adjusts  movement  formations  only  after  enemy  contact. 

Leader  conducts  spot-checks  to  insure  that  his  instructions  are  being  carried  out  only  after  inadvertently  finding  a 
discrepancy. _ 


_ Behavior _ Poor  Typical  Outstandin; 

Leader  continues  an  operation  regardless  of  the  chances  of  success  or  failure. _ _ _ 

Leader  continues  the  operation  “to  die  last  man”  and  does  not  inform  higher. _ 

Leader  continues  to  try  and  accomplish  the  assigned  mission  even  when  his  unit  sustains  too  many  casualties.  _ _ _ 

Leader  continues  with  an  operation  until  his  unit  is  not  combat  effective  and  then  informs  higher. _ 
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50  Leader  directs  continued  movement  of  his  force  in  same  manner  and  direction  even  while  receiving  enemy  fires, 

5 1  Leader  directs  cross  leveling  of  critical  resources  among  all  elements  when  one  element  should  have  priority  for 

_ the  resources, _ 

52  Leader  directs  fires  on  enemy  locations  outside  his  assigned  sector  without  coordination  with  his  higher  or 

adjacent  units.  _ _ 


Leader  directs  movement  of  his  force  away  from  incoming  enemy  fires. _ _ 

Leader  directs  movement  of  his  force  to  avoid  enemy  fires. _ 

Leader  directs  periodic  halts  during  movement  to  adjust  equipment/change  socks. 
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_ subordinate. _ _ _ _____ _ _ _ _ _ 

76  Leader  directs  the  replacement  of  a  critical  individual  who  has  become  a  casualty  only  after  prompting  by  a 

_ superior. _ 

77  Leader  directs  use  of  a  security  element,  but  does  not  specify  a  location/area  based  on  likely  enemy  location/area. 

(Offense) _ 
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93  Leader  displays  evidence  of  his  knowledge  of  the  enemy  capabilities  and  terrain  appreciation  by  informing 
subordinate  units  of  enemy  activities  in  an  area  from  which  they  are  capable  of  observing  unit  positions  or 

_ activities. _ _ 

94  Leader  displays  evidence  of  his  knowledge  of  the  enemy  capabilities  and  terrain  appreciation  by  informing 
subordinate  units  of  nearby  enemy  activities. 
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prioritize  specific  tasks. 

1 1 1  Leader  displays  evidence  of  his  lack  of  understanding  of  the  time  needed  for  completing  the  assigned  movement. 

1 12  Leader  displays  evidence  of  his  lack  of  understanding  of  the  time  needed  to  accomplish  assigned  tasks  during 

_ actions  on  the  objective. _ 


_ Behavior _ Poor  :  Ti 

1 1 3  Leader  displays  evidence  of  his  understanding  of  the  time  constraints  on  subordinate  units  by  prioritizing  the 

_ effort  to  be  expended  on  specific  tasks. _ 

1 14  Leader  displays  evidence  of  his  understanding  of  the  time  needed  for  completing  the  assigned  movement. 

1 15  Leader  displays  evidence  of  his  understanding  of  the  time  needed  for  subordinate  unit  preparation  by  providing 
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133  Leader  does  not  recognize  subordinate  physical  and/or  mental  exhaustion  until  a  soldier  needs  medical  attention. 

134  Leader  does  not  report  that  his  unit  is  at  a  danger  area  or  takes  no  action  to  avoid  it. _ 

135  Leader  does  not  take  action  to  establish  alternate  means  of  communication  when  the  primary  means  of 

_ communications  is  lost  with  a  subordinate.  _ 

136  Leader  does  not  warn  subordinates  of  civilian  movement  in  the  area. 


# _  Behavior  Poor  Typical  Outstanding 

137  Leader  ensures  that  all  appropriate  personnel  know  the  signals  that  are  about  to  be  used  during  an  operation, 

138  Leader  establishes  lateral  coordination  with  peers  or  adjacent  units. 

139  Leader  explains  how  to  execute  a  task  or  drill  that  his  unit  knows  and  subordinates  must  remind  him  that  they  are 
already  trained. 
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_ _ Behavior  Poor  Typical  Qiitstandinj 

159  Leader  fails  to  react  immediately  to  recommendations/advice  from  subordinates, _ _ 

160  Leader  fails  to  react  to  additional  intelligence  on  any  one  of  the  critical  factors  of  mission,  enemy,  terrain,  troops, 
time,  and  civilians. 

1 6 1  Leader  fails  to  react  to  an  enemy  attack  or  counterattack. 


GO  >>  Gft 

«3  <S  §  3 

te  te  & 

m  &  2  8 

0>  $> 
i-J  a— 3  ^3  i-J 


A-10 


179  Leader  issues  “be  prepared”  orders  only  after  receiving  guidance  from  higher. 

180  Leader  issues  “be  prepared”  orders  without  guidance  from  higher. 

181  Leader  keeps  support/attached  elements  informed  of  his  location  during  movement  by  ensuring  that  support 
elements  report  phase  lines  or  position  location  to  their  parent  units. 


_ Behavior  Poor  Typical  Outstandin 

Leader  maintains  friendly  battle  tracking  of  subordinate  combat  forces. 

Leader  modifies  his  current  plan/activity  in  order  to  accommodate  a  situation  that  is  evolving  by  an  adjacent 

friendly  unit  after  receiving  directions  to  do  so  from  a  higher  element. _ _ _ 

Leader  modifies  his  current  plan/activity  in  order  to  accommodate  a  situation  that  is  evolving  by  an  adjacent 
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202  Leader  orders  a  shift  of  indirect  fires  to  facilitate  friendly  force  action. 

203  Leader  periodically  requests  adjacent  unit  information  update  if  not  provided  by  higher  element. 

204  Leader  plans  for  the  effective  use  of  time  given  the  situation  and  circumstances. 

205  Leader  plans  personnel  rotation  to  have  best  personnel  at  the  appropriate  location  to  complete  critical  tasks. 


Behavior _  Poor  Typical  Outstandin; 

206  Leader  presents  a  plan  that  acknowledges  and  accommodates  the  time  constraints  imposed. 

207  Leader  presents  a  plan  that  will  not  accomplish  the  task  in  the  time  required. 

208  Leader  projects  future  possible  mission  for  his  unit  and  directs  a  “be  prepared”  order  to  subordinates,  without 

direction  from  higher  element.  _ 
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228  Leader  relays  the  observation  of  a  visual  signal  executed  during  the  current  operation  to  subordinate  units. 

229  Leader  relies  too  heavily  on  the  advice  of  others  for  decision  making. _ 

230  Leader  relocates  forces  to  block  anticipated  enemy  attack  or  counterattack. _ _ _ 

23 1  Leader  remains  calm  during  an  extremely  tense  situation  and  responds  in  a  timely  manner. 

232  Leader  reminds  subordinates  at  the  last  minute  of  approaching  time  or  location  for  a  critical  event. 
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Leader  reports  unit  movements  and  locations  using  the  graphic  references  from  the  current  operation. 
Leader  requests  a  shift  of  indirect  fires  to  facilitate  friendly  force  action  only  after  a  request  to  do  so  fr< 
subordinate  element. 
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contact  or  while  in  a  danger  area,  after  prompting  from  subordinate  or  higher  element, _ 

Leader  requests  unplanned  smoke  or  obscurants  to  provide  protection  for  his  unit  while  in  unexpected  enemy 

contact  or  while  in  a  danger  area,  _ _ _ 

Leader  retransmits  to  subordinates  almost  all  of  the  messages  he  receives  from  above  verbatim- _ 

Leader  rotates  tasks  among  subordinate  units  to  avoid  burnout. 
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Appendix  B 

Radio  Communications  Checklist  of  Leader  Awareness  (RCCOLA) 
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Radio  Communications  Checklist  of  Leader  Awareness 


Date _ 

Time_ 

Squad. 

Rater 


_ _ PLANNING  /  PREPARING _ 

OUTSTANDING  ~ 

_ requests  additional  time  or  assets  when  an  unrealistic  task  is  assigned. 

_ anticipates  noncombatant  actions  within  his  area  and  directs  elements  to  be  prepared  to  respond. 

TYPICAL  " 

_ directs  subordinates  to  conduct  communication  checks  before  mission  begins. 

_ conveys  an  accurate  picture  of  the  situation  after  answering  some  questions  from  subordinates. 

_ provides  warning  to  subordinate  leaders  of  a  change  in  mission  upon  notification  from  higher 

headquarters. 

_ directs  a  “be  prepared"  order  to  subordinates,  after  receiving  planning  directions  from  higher. 

POOR  ~  — 

_ fails  to  disseminate  or  inadequately  disseminates  critical  change-of-mission  information  or  factors 

impacting  current  mission  to  subordinates. 

_ fails  to  anticipate  the  need  for  night  observation  devices  and  begins  night  operations  without  them. 

_ does  not  notify  appropriate  personnel  of  the  meaning  of  signals  that  are  being  used  during  an 

operation. 

_ does  not  issue  “be  prepared”  orders  after  receiving  guidance  from  higher  to  do  so. 

_ does  not  convey  an  accurate  picture  of  the  situation  even  after  answering  questions. 

_ does  not  convey  a  complete  picture  of  the  situation  even  after  answering  questions. 

_ does  not  convey  the  commander's  intent  to  subordinates  during  a  change  of  mission  order. 

_ presents  a  plan  that  will  not  accomplish  the  task  in  the  time  required. 


MOVEMENT 

OUTSTANDING 

_ anticipates  activity  and  locates  himself  at  the  best  position  to  control  unit. 

_ reports  encountering  mines  or  obstacles  along  unit  route  of  movement  and  presents  the  operational 

impact  or  possible  COAs  to  overcome  the  impediment. 

TYPICAL 

_ occasionally  must  ask  subordinates  to  report  their  current  positions. 

_ directs  a  change  in  unit  movement  formation  because  terrain  just  encountered  has  changed, 

_ recognizes  that  his  unit  has  moved  into  a  danger  area,  he  reports  this,  and  then  takes  action  to  move 

through  or  out  of  the  danger  area. 

_ reports  encountering  mines  or  obstacles  along  his  unit  route  of  movement. 

_ modifies  plan  or  activity  to  accommodate  a  situation  evolving  in  an  adjacent  friendly  unit,  after  receiving 

directions  to  do  so  from  a  higher. 

POOR 

_ does  not  report  that  his  unit  is  at  a  danger  area  or  takes  no  action  to  avoid  it. 

_ displays  little  knowledge  of  the  enemy  capabilities  or  terrain,  failing  to  inform  subordinate  of  enemy 

_ activities  in  an  area  capable  of  observing  or  bringing  direct  fire  on  unit  positions  or  activities. _ 
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ACTIONS  ON  ENEMY  CONTACT 


OUTSTANDING 

_ when  asked  for  a  SITREP  while  actively  engaged  with  the  enemy,  can  immediately  respond  with  accurate 

information, 

_ correctly  identifies  weakest  enemy  point. 

_ directs  the  relocation  of  a  subordinate  element  to  be  prepared  to  assist/reinforce  an  expected  weakness  by  another 

friendly  element. 

_ requests  assets  to  augment  unit  to  assist  with  mission  accomplishment  before  unit  strength  becomes  inadequate 

to  accomplish  mission. 

_ displays  evidence  of  fire  control  measures  and  a  change  in  threat  or  danger  to  the  unit  by  recommending  the  lifting 

or  shifting  of  supporting  fires  in  an  adjacent  sector. 

_ plans  personnel  rotation  to  have  best  people  at  appropriate  locations  to  complete  critical  tasks. 

_ directs  subordinates  to  break  enemy  contact  because  cost  of  fighting  the  enemy  is  higher  than  the  benefit. 

_ displays  evidence  of  his  knowledge  of  the  enemy  or  friendly  situation  by  relieving  or  replacing  a  unit  or  element 

before  it  has  become  ineffective. 

_ directs  subordinate  to  take  an  action  that  distracts  the  enemy  from  the  friendly  unit  main  effort  or  action. 

_ requests  ammunition  resupply,  projecting  that  current  supplies  will  be  exhausted  in  30  minutes  given  the  present 

rate  of  expenditure. 

_ informs  higher  that  the  unit  is  nearing  a  status  of  non-combat  effective  early  enough  so  higher  can  react. 

_ presents  the  future  likelihood  of  threat  COAs  in  providing  SITREPs  to  the  higher  element  while  actively  engaged 

with  the  enemy. 

TYPICAL  - - 

_ reports  enemy  activity  in  his  area  to  the  higher  element. 

_ moves  forces  to  respond  to  an  enemy  attack  or  counterattack. 

_ directs  a  soldier  to  take  charge  of  an  element  when  he  is  informed  that  the  element  leader  is  a  casualty. 

_ directs  subordinates  to  continue  actions  because  the  mission  is  not  yet  fully  accomplished  or  complete. 

_ displays  evidence  of  his  knowledge  about  enemy  capabilities  and  terrain  by  informing  subordinate  units  of  nearby 

enemy  activities. 

POOR  “  - - 

_ fails  to  designate  a  new  element  leader  when  one  of  them  becomes  a  casualty. 

_ fails  to  direct  the  replacement  of  a  critical  individual  who  has  become  a  casualty. 

_ directs  that  no  subordinates  relocate  even  when  a  subordinate  element  notifies  him  that  assistance  or 

reinforcement  is  needed  to  accomplish  the  mission. 

_ directs  subordinates  to  halt  actions  before  the  mission  is  accomplished/complete  even  though  sufficient  resources 

are  available  to  continue  the  mission. 

_ continues  the  operation  "to  the  last  man”  and  does  not  inform  higher. 


MISCELLANEOUS 

OUTSTANDING  “ 

_ whenever  asked,  the  leader  can  immediately  provide  a  detailed  and  accurate  platoon  ACE  report. 

_ modifies  his  current  plan/activity  to  accommodate  a  situation  evolving  in  an  adjacent  friendly  unit. 

_ conveys  a  complete  picture  of  the  current  situation  to  his  subordinates. 

_ takes  action  that  is  beneficial  to  civilian  population  without  hindering  operations. 

TYPICAL  ~ - 

_ requests  medical  evacuation  for  an  injured  soldier. 

_ reacts  to  noncombatant  actions  in  the  area. 

_ uses  an  alternate  frequency  when  primary  frequency  fails  to  make  contact  with  intended  station. 

_ reports  a  change  of  command  post  location. 

_ notifies  appropriate  personnel  of  the  meaning  of  signals  being  used  during  an  operation. 

_ directs  a  change  in  MOPP  based  on  commander’s  guidance,  orders,  or  the  SOP. 

POOR  — 

_ does  not  notify  subordinates  of  a  friendly  ground  unit  moving  through  the  area,  which  could  lead  to  fratricide. 

_ does  not  warn  subordinates  of  civilian  movement  in  the  area. 

_ treats  a  sound  recommendation  or  advice  from  subordinates  as  an  interruption  and  takes  no  action. 

_ does  not  notify  subordinates  of  a  friendly  aircraft  moving  through  the  area,  which  could  lead  to  fratricide. 

_ fails  to  direct  any  reallocation  of  critical  resources 
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Future  Expectations  of  Likely  Leader  Awareness 


Date  _ 

Time  _ 

Squad  _ 

Rater  _ 

Directions.  Based  on  the  squad  and  platoon  radio  communication  you  heard  during  the  last 
mission,  rate  the  Squad  Leader's  (SL's)  likelihood  of  exhibiting,  in  most  kinds  of  future  missions, 
the  various  behaviors  listed  below.  Use  the  following  scale  to  make  your  ratings: 

1.  highly  unlikely 

2.  unlikely 

3.  somewhat  doubtful 

4.  hard  to  say 

5.  a  slight  chance 

6.  likely 

7.  highly  likely 

Select  the  choice  which  best  meets  with  your  future  expectation  of  each  behavior  and  enter  it  in 
the  blank  to  the  left  of  each  item. 


_  This  SL  could  be  expected  to  direct  squad  members  to  conduct  communication  checks 

before  future  missions. 

_  When  asked  for  a  SITREP  while  actively  engaged  with  the  enemy,  this  SL  could  be 

expected  to  immediately  respond  with  accurate  information. 

_  This  SL  could  be  expected  not  to  notify  appropriate  personnel  of  the  meaning  of  signals 

that  are  being  used  during  an  operation. 

_  This  SL  could  be  expected  to  direct  squad  members  to  continue  their  actions  if  the 

mission  is  not  yet  fully  accomplished  or  complete. 

_ This  SL  could  be  expected  not  to  notify  squad  members  of  a  friendly  aircraft  moving 

through  the  area,  which  could  lead  to  fratricide. 

_  This  SL  could  be  expected  to  inadequately  disseminate  or  fail  to  disseminate  to  squad 

members  mission-critical  change-of-mission  information  or  factors  impacting  current  mission. 

_  This  SL  could  be  expected  to  relocate  a  fire  team  or  buddy  team  to  be  prepared  to  assist 

or  reinforce  an  expected  weakness  by  another  friendly  element. 

_ This  SL  could  be  expected  to  use  an  alternate  frequency  when  the  primary  frequency  fails 

to  make  contact  with  the  intended  station. 
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_  This  SL  could  be  expected  not  to  report  being  in  a  danger  area  or  could  be  expected  to 

take  no  action  to  avoid  it. 

_  This  SL  could  be  expected  to  report  a  change  in  his  location. 

_  This  SL  could  be  expected  to  continue  an  operation  "to  the  last  man"  without  informing 

the  Platoon  Leader. 

_  This  SL  could  be  expected  to  request  medical  evacuation  for  an  injured  soldier, 

_  This  SL  could  be  expected  to  display  evidence  of  his  knowledge  of  enemy  capabilities 

and  terrain  by  informing  the  squad  of  nearby  enemy  activities. 

_  This  SL  could  be  expected  to  present  a  plan  that  will  not  accomplish  the  task  in  the  time 

required. 

_  This  SL  could  be  expected  to  report  encountering  mines  or  obstacles  along  the  squad's 

route  of  movement  and  to  present  the  operational  impact  or  possible  courses  of  action  to 
overcome  the  impediment. 

_  This  SL  could  be  expected  to  take  action  that  is  beneficial  to  the  civilian  population 

without  hindering  his  squad  operations. 

_  This  SL  could  be  expected  to  fail  to  direct  the  replacement  of  a  critical  individual  who 

has  become  a  casualty. 

_  This  SL  could  be  expected  not  to  issue  "be  prepared"  orders  after  receiving  guidance 

from  the  platoon  to  do  so. 

_  This  SL  could  be  expected  to  inform  the  Platoon  Leader  that  his  squad  is  nearing  a  non¬ 
combat  effective  status  early  enough  so  the  platoon  can  react  appropriately. 

_  This  SL  could  be  expected  to  report  enemy  activity  in  his  area  to  the  platoon. 

_  This  SL  could  be  expected  to  convey  an  accurate  picture  of  the  situation  after  answering 

some  questions  from  team  leaders  or  squad  members. 

_  This  SL  could  be  expected  to  fail  to  designate  a  new  fire  team  leader  when  one  has 

become  a  casualty. 

_ This  SL  could  be  expected  to  issue  "be  prepared"  orders  to  the  squad,  after  receiving 

planning  directions  from  the  Platoon  Leader. 
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_ After  receiving  directions  from  the  Platoon  Leader,  this  SL  could  be  expected  to  modify 

his  current  plan  or  activity  in  order  to  accommodate  an  evolving  situation  in  an  adjacent  friendly 
unit. 


_  This  SL  could  be  expected  to  convey  a  complete  picture  of  the  current  situation  to  his 

squad. 


_ This  SL  could  be  expected  to  recognize  his  unit  has  moved  into  a  danger  area,  to  report 

this,  and  to  take  action  to  move  through  or  out  of  the  danger  area. 

_  This  SL  could  be  expected  to  display  evidence  of  fire  control  measures  and  to  recognize  a 

change  in  threat  or  danger  to  his  squad  by  recommending  the  lifting  or  shifting  of  fires  in  an 
adjacent  sector. 

_ This  SL  could  be  expected  to  report  encountering  mines  or  obstacles  along  the  squad's 

route  of  movement. 

_ This  SL  could  be  expected  to  correctly  identify  the  weakest  enemy  point. 

_  This  SL  could  be  expected  not  to  convey  an  accurate  picture  of  the  situation,  even  after 

answering  questions. 

_  This  SL  could  be  expected  not  to  convey  the  commander's  intent  to  squad  members 

during  a  change-of-mission  order. 

_ This  SL  could  be  expected  to  direct  his  squad  to  take  an  action  that  distracts  the  enemy 

from  the  friendly  unit  main  effort  or  action. 

_  This  SL  could  be  expected  to  request  ammunition  resupply,  projecting  that  squad 

supplies  will  be  exhausted  in  30  minutes  given  the  present  rate  of  expenditure. 

_  This  SL  could  be  expected  to  direct  a  change  in  MOPP  based  on  commander's  guidance, 

orders,  or  the  SOP. 

_  This  SL  could  be  expected  to  anticipate  noncombatant  actions  in  his  area  and  to  direct  his 

squad  to  be  prepared  to  respond. 

_ This  SL  could  be  expected  not  to  warn  his  squad  of  civilian  movement  in  the  area. 

_  This  SL  could  be  expected  to  anticipate  activity  and  to  locate  himself  at  the  best  position 

to  control  his  squad. 

_ This  SL  could  be  expected  not  to  notify  squad  members  of  a  friendly  ground  unit  moving 

through  the  area,  which  could  lead  to  fratricide. 
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_  This  SL  could  be  expected  to  display  evidence  of  his  knowledge  of  the  enemy  or  friendly 

situation  by  relieving  or  replacing  squad  members  before  they  become  ineffective. 

_ This  SL  could  be  expected  to  react  to  noncombatant  actions  in  his  area. 

_  This  SL  could  be  expected  to  direct  a  change  in  the  squad's  movement  formation  because 

the  terrain  they  just  encountered  has  changed. 

_  This  SL  could  be  expected  not  to  relocate  personnel  when  one  team  notifies  him  that 

assistance  or  reinforcement  is  needed  to  accomplish  the  mission. 

_ Whenever  asked,  this  SL  could  be  expected  to  immediately  provide  a  detailed  and 

accurate  ACE  report. 

_  hi  providing  SITREPs  to  the  Platoon  Leader  while  actively  engaged  with  the  enemy,  this 

SL  could  be  expected  to  present  die  future  likelihood  of  threat  CO  As. 

_  This  SL  could  be  expected  to  halt  squad  actions  before  the  mission  is  accomplished  or 

complete,  even  though  sufficient  resources  are  available  to  continue  the  mission. 

_ This  SL  could  be  expected  to  move  personnel  to  respond  to  an  enemy  attack  or 

counterattack. 

_  This  SL  could  be  expected  to  fail  to  anticipate  the  need  for  night  observation  devices  and 

to  begin  night  operations  without  them. 

_  This  SL  could  be  expected  to  notify  appropriate  personnel  of  the  meaning  of  signals 

currently  being  used  during  an  operation. 

_  This  SL  could  be  expected  to  direct  a  soldier  to  take  charge  of  a  fire  team  when  informed 

its  leader  is  a  casualty. 

_ This  SL  could  be  expected  to  request  additional  time  or  assets  when  an  unrealistic  task  is 

assigned. 

_  This  SL  could  be  expected  to  display  little  knowledge  of  enemy  capabilities  or  terrain,  by 

failing  to  inform  his  squad  of  enemy  activities  in  an  area  where  they  are  capable  of  observing  or 
bringing  direct  fire  upon  squad  positions  or  activities. 

_  This  SL  could  be  expected  to  modify  his  current  plan  or  activity  in  order  to  accommodate 

an  evolving  situation  in  an  adjacent  friendly  unit. 

_ This  SL  could  be  expected  to  provide  a  change-of-mission  warning  to  his  fire  team 

leaders  upon  notification  from  the  platoon. 
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_  This  SL  could  be  expected  to  occasionally  ask  squad  members  to  report  their  current 

positions. 

_ This  SL  could  be  expected  to  plan  personnel  rotations  to  have  the  best  soldiers  at  the  right 

locations  to  complete  critical  tasks. 

_  This  SL  could  be  expected  to  treat  sound  advice  from  his  team  leaders  as  an  interruption 

and  to  take  no  action  on  their  recommendations. 

_  This  SL  could  be  expected  to  direct  subordinates  to  break  enemy  contact  because  the  cost 

of  fighting  the  enemy  is  higher  than  the  benefit. 

_  This  SL  could  be  expected  to  request  platoon  assets  to  augment  his  squad  before  its 

strength  becomes  inadequate  to  accomplish  the  mission. 

_ This  SL  could  be  expected  to  fail  to  direct  any  reallocation  of  critical  resources. 

_ This  SL  could  be  expected  not  to  convey  a  complete  picture  of  the  situation,  even  after 

answering  questions. 


a******************************************************************,,.***.,,*,,,,,,,,,,,.,,, 

Overall,  how  would  you  expect  this  SL's  level  of  situational  awareness  to  be  in  future  missions, 
relative  to  other  SLs? 

_  Outstanding 

_  Well  above  peers 

_  Slightly  above  the  norm 

_  Center  of  mass 

_  Slightly  below  the  norm 

_ Well  below  peers 

_ Poor 

****************************************************************************** 
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Observation  Frequency  of  RCCOLA  Items  by  Two  Raters  Over  42  Experimental  Trials 
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Radio  Communications  Checklist  of  Leader  Awareness 


Date_ 

Time_ 

Squad. 

Rater 


_ _ PLANNING  /  PREPARING _ 

OUTSTANDING  " 

_ 0_  requests  additional  time  or  assets  when  an  unrealistic  task  is  assigned. 

_ 4.  anticipates  noncombatant  actions  within  his  area  and  directs  elements  to  be  prepared  to  respond. 

TYPICAL 

56  directs  subordinates  to  conduct  communication  checks  before  mission  begins. 

29  conveys  an  accurate  picture  of  the  situation  after  answering  some  questions  from  subordinates. 

64  provides  warning  to  subordinate  leaders  of  a  change  in  mission  upon  notification  from  higher 
headquarters. 

12  directs  a  “be  prepared"  order  to  subordinates,  after  receiving  planning  directions  from  higher. 

POOR  " 

45  fails  to  disseminate  or  inadequately  disseminates  critical  change-of-mission  information  or  factors 
impacting  current  mission  to  subordinates. 

_ 3.  fails  to  anticipate  the  need  for  night  observation  devices  and  begins  night  operations  without  them. 

_ 0_  does  not  notify  appropriate  personnel  of  the  meaning  of  signals  that  are  being  used  during  an  operation. 

_ 5_  does  not  issue  “be  prepared”  orders  after  receiving  guidance  from  higher  to  do  so. 

35  does  not  convey  an  accurate  picture  of  the  situation  even  after  answering  questions, 

52  does  not  convey  a  complete  picture  of  the  situation  even  after  answering  questions. 

14  does  not  convey  the  commander's  intent  to  subordinates  during  a  change  of  mission  order. 

_ Q_  presents  a  plan  that  will  not  accomplish  the  task  in  the  time  required. 


_ _ MOVEMENT _ 

OUTSTANDING 

_ 3_  anticipates  activity  and  locates  himself  at  the  best  position  to  control  unit. 

_ Q_  reports  encountering  mines  or  obstacles  along  unit  route  of  movement  and  presents  the  operational 

impact  or  possible  COAs  to  overcome  the  impediment. 

TYPICAL 

20  occasionally  must  ask  subordinates  to  report  their  current  positions. 

28  directs  a  change  in  unit  movement  formation  because  terrain  just  encountered  has  changed. 

37  recognizes  that  his  unit  has  moved  into  a  danger  area,  he  reports  this,  and  then  takes  action  to  move 
through  or  out  of  the  danger  area. 

_ 3_  reports  encountering  mines  or  obstacles  along  his  unit  route  of  movement. 

_ 9_  modifies  plan  or  activity  to  accommodate  a  situation  evolving  in  an  adjacent  friendly  unit,  after  receiving 

directions  to  do  so  from  a  higher. 

POOR 

62  does  not  report  that  his  unit  is  at  a  danger  area  or  takes  no  action  to  avoid  it. 

17  displays  little  knowledge  of  the  enemy  capabilities  or  terrain,  failing  to  inform  subordinate  of  enemy 
_ activities  in  an  area  capable  of  observing  or  bringing  direct  fire  on  unit  positions  or  activities. _ 


D-2 


ACTIONS  ON  ENEMY  CONTACT 


OUTSTANDING  - - 

_41_  when  asked  for  a  SITREP  while  actively  engaged  with  the  enemy,  can  immediately  respond  with  accurate 
information. 

_ !_  correctly  identifies  weakest  enemy  point. 

_ 3_  directs  the  relocation  of  a  subordinate  element  to  be  prepared  to  assist/reinforce  an  expected  weakness  by  another 

friendly  element. 

_ Q-  requests  assets  to  augment  unit  to  assist  with  mission  accomplishment  before  unit  strength  becomes  inadequate  to 

accomplish  mission. 

_ 0.  displays  evidence  of  fire  control  measures  and  a  change  in  threat  or  danger  to  the  unit  by  recommending  the  lifting 

or  shifting  of  supporting  fires  in  an  adjacent  sector. 

_ Q_  plans  personnel  rotation  to  have  best  people  at  appropriate  locations  to  complete  critical  tasks. 

_ 5.  directs  subordinates  to  break  enemy  contact  because  cost  of  fighting  the  enemy  is  higher  than  the  benefit. 

_ £L  displays  evidence  of  his  knowledge  of  the  enemy  or  friendly  situation  by  relieving  or  replacing  a  unit  or  element 

before  it  has  become  ineffective. 

_ Q_  directs  subordinate  to  take  an  action  that  distracts  the  enemy  from  the  friendly  unit  main  effort  or  action. 

_ £L  requests  ammunition  resupply,  projecting  that  current  supplies  will  be  exhausted  in  30  minutes  given  the  present 

rate  of  expenditure. 

_ Q_  informs  higher  that  the  unit  is  nearing  a  status  of  non-combat  effective  early  enough  so  higher  can  react. 

_ 0_  presents  the  future  likelihood  of  threat  COAs  in  providing  SITREPs  to  the  higher  element  while  actively  engaged 

with  the  enemy. 

TYPICAL  ~ 

155  reports  enemy  activity  in  his  area  to  the  higher  element. 

19  moves  forces  to  respond  to  an  enemy  attack  or  counterattack. 

_ Q_  directs  a  soldier  to  take  charge  of  an  element  when  he  is  informed  that  the  element  leader  is  a  casualty. 

13  directs  subordinates  to  continue  actions  because  the  mission  is  not  yet  fully  accomplished  or  complete. 

215  displays  evidence  of  his  knowledge  about  enemy  capabilities  and  terrain  by  informing  subordinate  units  of  nearby 
enemy  activities. 

POOR  "  - - 

_ 0.  fails  to  designate  a  new  element  leader  when  one  of  them  becomes  a  casualty. 

_ 0_  fails  to  direct  the  replacement  of  a  critical  individual  who  has  become  a  casualty. 

_ 0.  directs  that  no  subordinates  relocate  even  when  a  subordinate  element  notifies  him  that  assistance  or 

reinforcement  is  needed  to  accomplish  the  mission. 

_ L  directs  subordinates  to  halt  actions  before  the  mission  is  accomplished/complete  even  though  sufficient  resources 

are  available  to  continue  the  mission. 

_ Q_  continues  the  operation  “to  the  last  man”  and  does  not  inform  higher. 


MISCELLANEOUS 

OUTSTANDING 

_ 2_  whenever  asked,  the  leader  can  immediately  provide  a  detailed  and  accurate  platoon  ACE  report. 

_ Q_  modifies  his  current  plan/activity  to  accommodate  a  situation  evolving  in  an  adjacent  friendly  unit. 

15  conveys  a  complete  picture  of  the  current  situation  to  his  subordinates. 

_ Q_  takes  action  that  is  beneficial  to  civilian  population  without  hindering  operations. 

TYPICAL 

_ 0_  requests  medical  evacuation  for  an  injured  soldier. 

_ 8_  reacts  to  noncombatant  actions  in  the  area. 

_ 2_  uses  an  alternate  frequency  when  primary  frequency  fails  to  make  contact  with  intended  station. 

Ill  reports  a  change  of  command  post  location. 

50  notifies  appropriate  personnel  of  the  meaning  of  signals  being  used  during  an  operation. 

_ 2_  directs  a  change  in  MOPP  based  on  commander’s  guidance,  orders,  or  the  SOP. 

POOR  ~~  “ 

_ L  does  not  notify  subordinates  of  a  friendly  ground  unit  moving  through  the  area,  which  could  lead  to  fratricide. 

_ 1_  does  not  warn  subordinates  of  civilian  movement  in  the  area. 

_ Q_  treats  a  sound  recommendation  or  advice  from  subordinates  as  an  interruption  and  takes  no  action. 

_ Q_  does  not  notify  subordinates  of  a  friendly  aircraft  moving  through  the  area,  which  could  lead  to  fratricide. 

_ 0_  fails  to  direct  any  reallocation  of  critical  resources 
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